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Introduction 



The Assessment of Achievement Programme 
(AAP) was established by the Scottish Office Edu- 
cation and Industry Department (SOEID) in 1981 
to monitor the performance of pupils in Scottish 
schools in particular areas of the curriculum. Since 
1983, there have been regular surveys in three core 
curricular areas — English language, mathematics 
and science. The main AAP objectives are to de- 
scribe national levels of attainment and to provide 
evidence about changes in these levels over time. 
The surveys are intended to inform the SOEID, 
education authorities, teachers and other interested 
parties about the achievement of pupils and to in- 
dicate ways of improving teaching and learning. 

The 1996 AAP Science survey 

This survey was directed by Rae Stark and Tom 
Bryce, with Donald Gray and Lynn Ferguson, at 
the Jordanhill Campus of the University of Strath- 
clyde. The directors were also involved in the three 
previous science surveys (1987, 1990 and 1993). 

Responsibility for drawing samples of schools and 
pupils lies with the AAP Central Support Unit 
(CSU) at the Scottish Council for Research in Edu- 
cation (SCRE) in Edinburgh. They advise on tech- 
nical matters and liaise with participating schools, 
distribute materials and carry out the statistical 
analysis of survey data. 

The main objectives of the 1996 AAP were: 

• to assess what P4, P7 and S2 pupils know 
and can do in agreed aspects of science 

• to relate performance, where possible, to lev- 
els in National Guidelines: Environmental 
Studies 5-14 (science component) 

• to provide comparisons, where possible, 
of performance in 1990, 1993 and 1996 



• to provide comparisons of performance 
across stages 

• to compare performance of girls and boys at 
each stage 

• to compare progress from P4 to P7 within 
the longitudinal sample. 

Sampling 

Samples of pupils at P4, P7 and S2 were selected 
to be representative of pupils in all mainstream 
schools whether education authority, grant-aided or 
independent. Approximately 6,500 pupils took part 
with a similar number of boys and girls being as- 
sessed on each aspect. Both written and practical 
forms of assessment were used. All pupils tackled 
the written component; sub-samples were drawn 
to undertake the practical component. 

This was the first AAP survey to include a longitu- 
dinal element: a number of P4 pupils involved in 
the 1993 survey were included in the 1996 P7 sam- 
ple. 

The assessment framework 

The assessment framework was developed to take 
account of the science component of National 
Guidelines for Environmental Studies (ES 5-14). 
The five Strands, Knowledge and Understanding 
(KU), Planning (P), Collecting Evidence (CE), 
Recording and Presenting (RP) and Interpreting 
and Evaluating (IE), provided the category head- 
ings and a number of sub-categories were identi- 
fied for the purposes of the AAP analysis. 

The specific content assessed and the contexts for 
skills tasks were drawn from the three Attainment 
Outcomes (AOs): Living Things and the Processes 
of Life, Energy and Forces and Earth and Space. 



1996 Assessment framework 



Knowledge and Understanding (KU) 

Planning (P) 

PI: Question raising 

P2: Identifying information sources and resources 

P3: Sequencing plans 

P4: Planning for recording and reporting 

P5: Anticipating problems 

P6: Planning for safety and hygiene 

Collecting Evidence (CE) 

CE 1 : Recognising similarities and differences 

CE2: Recognising changes 

CE3: Extracting information 

CE4: Using simple techniques 

CE5: Estimating and measuring 

CE6: Collecting evidence fairly and safely 

Recording and Presenting (RP) 

RP 1 : Recording in a variety of formats 
RP2: Presenting in a variety of formats 

Interpreting and Evaluating (IE) 

lEl : Indentifying relationships 
IE2: Evaluating evidence 



The assessment materials 

The nature of each individual Strand and Target 
determined how chosen tasks were to be assessed. 
Written tests were used wherever they were thought 
to be appropriate — Knowledge and Understand- 
ing was almost exclusively assessed through writ- 
ten tasks. In other instances practical tests were 
considered the only reliable and valid strategy. The 
survey therefore contained a significant practical 
component as well as written tests. 

Each test booklet contained tasks reflecting two or 
more 5-14 Levels to tap the range of attainment 
anticipated within each stage assessed. Five writ- 
ten test booklets were developed for each stage and 
any one pupil tackled two booklets. Two practical 
circuits were developed, each containing eight sta- 
tions and any one pupil tackled one circuit. Two 
practical investigations were used in the survey on 
a one-to-one basis; each pupil tackled one investi- 
gation. 

^ \ three-part questionnaire was included for pupils. 




The first section attempted to determine pupils’ 
confidence in their understanding in a range of top- 
ics and to identify their sources of learning. The 
second sought information on children’s prefer- 
ences for a selection of topics drawn from the three 
Attainment Outcomes. The third focused on learn- 
ing activities they might have experienced in school. 

Measuring performance 

In the 5-14 programme, pupils can be expected to 
attain the following levels of performance. 

Level A — attainable by almost all PI -P3 pup- 
ils. 

Level B — attainable by some P3 pupils, or 

even earlier, but certainly by most in 
P4. 

Level C — attainable by most P4-P6 pupils. 
Level D — attainable by some P5-P6 pupils, or 
even earlier, but certainly by most in 
P7. 

Level E — attainable by some P7-S1 pupils but 
certainly by most in S2. 
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For the purposes of this report, ‘most’ pupils has 
been set at 75 per cent of the sample at any stage. 
Each task was marked and subsequently analysed 
to give an index of achievement for ‘most pupils’ . 
This equated to the number of sub-tasks correctly 
completed and was expressed as a score lying be- 
tween 0 and 1 , where 0 indicates no grasp of the 5- 
14 target and 1 indicates a very secure grasp. Sat- 
isfactory attainment of a target was set at ‘most 
pupils’ achieving a score of > 0.8 on any individual 
task. 

The Strand Knowledge and Understanding differ- 
entiates between ‘secure’ knowledge and ‘making 
steady progress’. At any stage in an individual pu- 
pil’s development, it is likely that some concepts 
and principles will be well-established, others may 
be less secure or even absent. This survey indi- 
cates a ‘secure grasp’ of relevant knowledge by an 
index score > 0.8 and ‘making steady progress’ is 
indicated in scores between 0.35 and 0.8 



Table 1 : Modes of assessment 



MODE 


MATERIALS 


TIME ALLOWED 


Written 


illustrated booklet 


P4 - 30 mins 
(12 tasks per pupil) 
P7 - 45 mins 
(20 tasks per pupil) 
S2 - 60 mins 
(25 tasks per pupil) 


Practical 

Circuits 


apparatus & materials (stations circuits) 
illustrated booklet 


3-4 minutes per station 
(8 stations per pupil) 


Practical 

Investigations 


apparatus & materials (semi-structured tasks) 
illustrated booklet 


P4 - 20 mins 
(1 task per pupil) 
P7 - 20 mins 
(1 task per pupil) 
S2 - 30 mins 
(1 task per pupil) 




a 



3 



1 Knowledge 
and understanding 



The specific content which pupils should acquire 
and develop is set out in the three Attainment Out- 
comes: 

AOl Understanding Living Things and the Proc- 
esses of Life \ 

A02 Understanding Forces and Energy; 

AOS Understanding Earth and Space. 

The content statements within each Attainment Out- 
come are set out in three (st)age related bands: PI- 
PS (5-8 years); P4-P6 (8- 1 1 years); and P7-S2 (11- 
14 years). These statements map out the knowl- 
edge which pupils are expected to acquire and also 
provide the contexts in which a range of practical 
and process skills can be developed. 

For any one stage, the majority of tasks expected 
pupils to be able to demonstrate ‘secure knowledge 








and understanding’ of the relevant concepts. As 
many tasks were common to more than one stage, 
comparisons of performance levels across P4, P7 
and S2 could be made. 

The Tasks 

All tasks for Knowledge and Understanding came 
in two main forms. In the first, boxes had to be 
ticked or labels selected; in the second pupils were 
asked to explain an instance of a concept, demon- 
strating their understanding of the situation. The 
example Seasons is from the second. 

This task was developed to assesses a target in the 
PI - P3 band from Attainment Outcome 1 Living 
Things and the Processes of Life. It was included 
in the P4 and P7 written papers and required pupils 
to demonstrate an understanding of seasonal cam- 
ouflage. 

For most pupils the index of achievement at P4 was 
0.52 and at P7 was 0.74. Neither group demon- 
strated a secure grasp of relevant concepts. 

It should be noted that a substantial number of P4 
pupils made no response to tasks which required 
extended responses. The language burden of such 
tasks may well have depressed performance lev- 
els. 

Primary 4 Knowledge and 
Understanding 

A total of 33 tasks were included in the P4 written 
papers. Table 2 (next page) shows the distribution 
of stage bands. Attainment Outcome and indicates 
the numbers which showed ‘secure grasp’ and 
‘steady progress’. 

In order to have attained Level B, in line with 5-14 
criteria, most P4 pupils should have demonstrated 
a secure grasp of P1-P3 statements and be making 



Q 



Table 2: P4 written tasks 





AOl A02 A03 


No. 

of tasks 


secure 

grasp 


steady 

progress 


No. 

of tasks 


secure 
• grasp 


steady 

progress 


NO. 

of tasks 


secure 

grasp 


steady 

progress 


PI -3 


7 


4 


3 


3 


1 


2 


8 


7 


1 


P4-6 


5 


2 


2 


5 


1 


4 


2 


1 


1 


P7-S2 


2 


0 


2 








1 


0 


1 



steady progress on tasks assessing P4-P6 state- 
ments. Overall, P4 pupils demonstrated a secure 
grasp ofthe relevant knowledge in 12 of the 18 PI- 
PS tasks. In the remainder, there was clear evi- 
dence of some understanding of the concepts. In 
the other two stage bands most pupils were mak- 
ing steady progress. 

Primary 7 Knowledge and 
Understanding 

A total of 54 tasks assessed Knowledge and Un- 
derstanding at P7. Table 3 shows the distribution 
of ‘secure grasp’ and ‘steady progress’. 

To attain Level D, most P7 pupils should have dem- 
onstrated a secure grasp of the P 1 -P6 statements 
and have made steady progress on the tasks assess- 
ing P7-S2 statements. Overall, P7 pupils showed a 
secure grasp of most relevant P1-P3 statements 
within Attainment Outcomes 1 and 3, Living Things 



and the Processes of Life and Earth and Space. In 
two of the A02 tasks. Energy and Forces, most 
pupils showed a secure grasp of concepts assessed, 
but in the other two tasks, one score fell into the 
category of ‘making steady progress’ while the other 
demonstrated little understanding ofthe concepts. 

In tasks tapping P4-P6 statements, a majority 
showed a secure grasp, with the remainder making 
steady progress. All but two of the P7-S2 tasks 
reached the steady progress range, with a few show- 
ing a secure grasp. 

On this evidence, P7 pupils were close to the cut- 
off point for a ‘secure grasp’ on many of the tasks 
assessing P1-P6 statements and were clearly mak- 
ing progress on the P7-S2 concepts. Significant 
weaknesses in some tasks prevent the conclusion 
that P7 pupils have comprehensively attained Level 
D in this aspect ofthe 1996 survey. 



Table 3: P7 written tasks 





AOl A02 A03 


No. 

of tasks 


secu re 
grasp 


steady 

progress 


No. 

of tasks 


secure 

grasp 


steady 

progress 


No. 

of tasks 


secure 

grasp 


steady 

progress 


PI -3 


6 


4 


2 


4 


2 


1 


3 


3 


0 


P4-6 


9 


5 


4 


9 


6 


3 


7 


3 


4 


P7-S2 


5 


2 


2 


4 


0 


4 


7 


2 


4 
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Secondary 2 Knowledge and 
Understanding 

A total of 71 tasks were included in the S2 written 
papers. Table 4 details the performance levels at- 
tained by most pupils. 

To attain Level E, S2 pupils should have demon- 
strated a secure grasp of key features across PI -S2. 
The scores for tasks drawing on the statements in 
the P4-P6 band approached or achieved the secure 
grasp cut-off point of 0.8 in all AOl tasks and half 
of the A02 and A03 tasks. Figures for the P7-S2 
tasks demonstrated a secure grasp of the content 
on two of the 15 AOl tasks, four of the 16 A02 
tasks and eight of the 20 A03 tasks. All but seven 
of the remainder were within the ‘making steady 
progress’ band. 



Figures for S2 tasks indicate that most pupils had 
a secure grasp of the majority of relevant concepts 
in the P4 P6 band and were making steady progress 
on the P7-S2 content. One of the two tasks from 
the PI - P3 band posed considerable problems for 
pupils at both P7 and S2. On the evidence of the 
tasks in the survey the performance of most S2 
pupils showed only limited improvement on that 
of P7 pupils. Performance fell significantly short 
of the expected Level E. 

Comment 

These findings represent a good performance for 
pupils at P4 and a fair performance at P7, given 
the stage of implementation of the Environmental 
Studies 5-14 guidelines at the time of the survey. 
Performance of S2 pupils was poor given the spe- 
cific science teaching in years 1 and 2 of second- 
ary school. 



Table 4: S2 written tasks 





AOl A02 A03 




No. 

of tasks 


secure 

grasp 


steady 

progress 


No. 

of tasks 


secure 

grasp 


steady 

progress 


No. 

of tasks 


secure 

grasp 


steady 

progress 


Pl-3 








2 


0 


1 








P4-6 


4 


4 


0 


8 ^ 


4 


4 


6 


3 


3 


P7-S2 


15 


2 


12 


16 


4 


8 


20 


8 


10 




A 
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2 Planning 



Many of the targets within this Strand are set in a 
context which AAP cannot readily replicate, there- 
fore only two tasks assessing skills in planning in- 
vestigations were included in the written compo- 
nent of the 1996 survey. Both had been used in the 
1993 written papers and focused on the sub-cat- 
egory P3, Sequencing Plans. One task appeared in 
the P4 test booklets and both were used at P7. No 
Planning tasks were used at S2. 

The tasks 

The task Snail Salt (Level D/E) is an example of 
the type of task used. In this task, children had to 
outline a simple investigation to determine which 
substance, salt or water, inhibited a snail from pass- 
ing through a ring of salty water. They could re- 
spond by either writing about or drawing diagrams 
of the tests they would make. 



Example 2: Snail Salt 




The score for most P4 pupils was 0.32 . While some 
pupils were able to identify what should be done to 
test the two statements, most gave only partial re- 
sponses. 

The score for most P7 pupils was 0.61 . 

Comment 

There were too few tasks to make definitive state- 
ments on pupil performance within the ES 5-14 
Strand, Planning. This area does not readily lend 
itself to the formats of national monitoring. It is 
more appropriately assessed by the teacher within 
the context of ongoing classroom activities, cer- 
tainly with younger age groups. It is an important 
area however and one where, drawing on the lim- 
ited evidence available from these tasks and the 
Practical Investigations (Section 6), pupils do not 
seem to have a great deal of experience or exper- 
tise. 



3 Collecting Evidence 



Collecting Evidence, which incorporates skills of 
observing, handling information and investigating, 
formed the second largest group of tasks in the 1 996 
survey. Tasks included: recognising similarities and 
differences', changes over time', extraction of infor- 
mation from graphical formats', pupils' awareness 
of how to collect evidence fairly and safely. 

The tasks: written 

i. Recognising similarities and differences 
Pupils had to look closely at a drawing, match it to 
one of the groups shown and to provide a reason 
for their choice. They were given one point for the 
correct choice of group and an additional point if 
they gave a satisfactory explanation. 



Example 3: Leaves 




Most P4 pupils scored 0.57. 
Most P7 pupils scored 0.74. 



a. Extracting information 

Tasks in this sub-category assessed one aspect of 
information handling: extracting information from 
a range of formats. The range of graphical formats 
assessed was extended at P7 and S2 to include line 
graphs and diagrams. 



Example 4: Garden Pests 



Here is a table about garden pests. 



Name of pest 


What the pest 
eats 


What eats the 
pest 


greenfly 


tips and buds 


ladybirds 


whitefly 


leaves 


lacewings 


beetle grub 


roots 


birds/moles 


leaf miner 


leaves 


beetles 



Use the table to answer these questions. 

a. What eats beetle grubs? 

b. What eats greenfly? 

c. What part of the plant does the whitefly eat? 



Most P4 pupils scored 0.88, indicating that they 
experienced little difficulty. 

Most P7 pupils scored 0.95. 
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Hi. Collecting Evidence Fairly and Safely 
These tasks assessed pupils’ understanding of how 
to undertake practical tasks with attention to safety 
and hygiene and/or fair testing. 



Example 5: Cress 



Ann grew some cress seeds on cotton wool. 




Her friend said that cress seeds grow better on sawdust. 
Ann decided to find out if this is true by growing two 
trays of seeds - one on cotton wool and one on sawdust. 



To make this a fair test she should keep some things the 
same for each tray. 



Write down three things which you think she should keep 
the same. 



Here pupils were required to identify three aspects 
of the investigation which should be kept the same 
to ensure 'fair testing.' 

Most P4 pupils scored 0.43. Many pupils had some 
grasp of ‘fair testing’ but few were able to give a 
complete answer to the task. 

Most P7 pupils scored 0.73. 

The tasks: practical 

These tasks focused on observing similarities and 
differences, changes over time and using simple 
techniques for ‘doing science’. The example 
Smartie assessed pupils’ skills in observing simi- 
larities and differences as a result of a practical ac- 
tivity (CEl). 



Example 6: Smartie 



Follow these instructions carefully. 

1 . Take a piece of paper and lay it on the tray in front of you. 

2. Put a sweet on the piece of paper 

3. Put 8 drops of water on the sweet and watch carefully as it 
soaks into the paper 

- dropper 




paper 



4. Wait a few minutes. 

Look carefully at the colours on the paper. 

5. Look at the three cards labelled P, Q, R. 

6. Decide which of these patterns was also made with a brown 
sweet. 

Write the letter of the card here 

Why do you think so? 

Tidy up when you have finished 



Pupils followed a series of instructions to obtain a 
simple chromatogram from a sweet. They then had 
to match their pattern to one of three provided and 
to justify their decision. A maximum of two points 
was possible on this task — one for the correct 
match and one for an acceptable Justification. 

Most P4 pupils scored 0.67 and most P7 scored 
0.78. Many pupils at both stages were able to match 
the patterns but a smaller number Justified the de- 
cisions adequately. 

Primary 4 Collecting Evidence 

A total of 12 written and 1 3 practical tasks assessed 
performance on Collecting Evidence at P4. Table 
5 (overleaf) shows the numbers of these which were 
‘secure’ (i.e. with an index > 0.8 at each Level). 
The final column shows the mean index score 
across all the tasks at that Level and the range of 
scores attained by most pupils. 

To meet 5-14 expectations, P4 pupils should have 
attained at least 0.8 on Level B and be making 
progress on Level C tasks. While only a small 
number of tasks reached the ‘secure’ threshold, the 
range of scores indicated that most pupils were 
performing reasonably well on many of the tasks. 
However examination of individual tasks indicated 
that main strengths lay in handling information. 
Performance was relatively weak on ‘fair testing’ 
and observing tasks. 



Table 5: P4 written and practical tasks 





No. of tasks 


No. "secure" 


Mean score 


Score range 


written 


1 


1 


0.98 




practical 


5 


1 


0.73 


0.69 - 0.83 


written 


1 1 


4 


0.66 


0.43 • 0.96 


practical 


8 


2 


0.69 


0.49 • 0.88 



Commeoit 

Performance was good at S2 with pupils operating 
at close to the expected 5- 1 4 Level. P4 and P7 were 
not yet secure at the relevant Levels with consider- 
able variation in performance on tasks. The more 
traditional ‘handling information’ tasks were suc- 
cessfully tackled at all three stages and in line with 
5-14 expectations. 



Primary 7 Coiiecting Evidence 

A total of 24 written and 13 practical tasks were 
included at P7. 

To meet 5- 14 expectations, P7 should have attained 
at least 0.8 on Level B, C and D tasks and be show- 
ing progress on Level E tasks. Mean scores for most 
P7 pupils indicated that they attained the cut-off 
score for a secure grasp at Levels B and C. How- 
ever, a lower number met this criterion for the writ- 
ten tasks at Level C. As at P4, ‘fair testing’ and 
observing were the weakest areas. Level D perform- 
ance did not meet expectations although all but one 
of the tasks (0.44) were within an acceptable range. 
Performance on Level E tasks was satisfactory. 



Table 6: P7 written and practical tasks 





No. of tasks 


No. "secure" 


Mean score 


Score range 


practical 


5 


5 


0.85 


0.81 - 0.96 


written 


15 


8 


0.81 


0.59 - 0.99 


practical 


8 


6 


0.86 


0.78 - 0.95 


written 


5 


2 


0.70 


0.44 - 0.84 


written 


4 


2 


0.69 


0.33 - 0.93 



Such skills can be assessed by written tests, which 
have been a feature of all four science surveys to 
date, but practical contexts should also be used to 
ensure validity of the assessment. Requisite skills 
are best acquired and developed through practical 
activities which are purposeful and demand an in- 
tegration of skills, processes and conceptual under- 
standing. 



Table 7: S2 written and practical tasks 



Level B 
Level C 



Level D 





No. of tasks 


No. "secure" 


Mean score 


Score range 


practical 


2 


1 


0.87 


0.72 - 0.83 


written 


12 


1 1 


0.88 


0.61 - 0.98 


practical 


9 


7 


0.84 


0.53-0.97 


written 


5 


4 


0.82 


0.55 -0.92 


practical 


3 


3 


0.87 


0.80 - 0.98 


written 


7 


6 


0.81 


0.61 - 0.93 



Secondary 2 Collecting Evidence 

A total of 38 tasks were used at S2, 24 written and 
14 practical. 

To meet 5-14 expectations, S2 pupils should have 
attained 0.8 at all Levels. All mean scores reached 
the cut-off figure and, overall, pupils demonstrated 
a secure grasp of the majority of the tasks. Per- 
formance on a few tasks was less than satisfactory 
and again these tended to be those assessing the 
observing and ‘fair testing’ statements. 
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4 Recording 
and Presenting 



This Strand contains two broad groups of state- 
ments, only one of which — recording in a variety 
of formats — was assessed in the survey. The sec- 
ond group — including oral presentations, personal 
topic folders and displays — was not appropriate 
to assessment in national monitoring exercises. The 
number of tasks representing this Strand was rela- 
tively small and findings should be considered 
alongside those of the AAP Mathematics surveys. 

Table 9: Number of written and practical RPJ 
tasks by stage 





P4 


P7 


52 


Written 


Practical 


Written 


Practical 


Written 


Practical 


RPl 


5 


2 


4 


2 


8 


2 



The tasks: written 

At the primary stages, most of the tasks required 
pupils to complete tables and graphs of different 
kinds using the information given. The range of 
formats was extended to include line graphs for P7 
and S2. Example 7 Teacup was common to both 
stages and the inclusion of a line graph sets the task 
at Level D. The ES 5-14 guidelines expect pupils 
to provide their own labels and axes. 

Assessment was based on of accuracy of points, 
correct format (line), and appropriate form of line 
to join the points (curve or series of short lines). 

Most P7 pupils scored 0.58 — the relatively low 
score can be attributed partly to the tendency to 
Join points in a series of short lines rather than an 
appropriate curve. 

Most S2 scored 0.76. 
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Example 7: Teacup 



The temperature of a cup of tea was taken every minute as it 
cooled down. 



Time (min) 


0 


1 


2 


3 


4 


Temp ('C) 


71 


68 


66 


65 


64 



Use the information to complete the line graph. 

The first two points have been put in for you. Put in the rest 
and join them up. 




Time (min) 



The tasks: practical 

Primary pupils were asked to sort objects into 
groups and to complete a table and a graph to show 
the number in each group as in Example 8 (over). 

Pupils had to count the different types of nuts, bolts 
etc. and to construct a bar graph to display their 
findings. 

Most P4 pupils scored 0.75. 

Most P7 pupils scored 0.92. 

The two S2 practical tasks assessed aspects of 
‘drawing diagrams using conventional symbols’ 
which were considered inappropriate for earlier 
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stages. In one task, pupils had to construct a cir- 
cuit from a drawing and then draw the circuit dia- 
gram. In the second, they filtered muddy water 
and, while the mixture was trickling through, drew 
a diagram of the assembled apparatus. In each case, 
the diagram was the focus of assessment within this 
category. Both tasks were Level E. 

Example 8: Nuts 



Look at the objects in front of you. 
Sort them into 4 groups. 



Table 10: P4 summary performance 



bolts 



washers 




0 ® 




There are two parts to this task. 

1 Complete this table to show how many of each you found. 



bolts 


nuts 


nails 


washers 











2. Use the numbers in the table to draw a bar chart here. 
Remember to label the axes. 



Put the objects back in the box before you leave the station. 



Primary 4 Recording and 
Presenting 

There were a total of seven Recording and Pre- 
senting tasks, five written and two practical. 

To meet 5-14 expectations, P4 pupils should have 
reached 0.8 on Level B tasks and shown some grasp 
of Level C tasks. Most pupils met the criteria but 
the limited number of tasks on this Strand makes it 
difficult to draw overall conclusions. 



Level B 
Level C 
Level D 





No. of tasks 


No. "secure" 


Mean score 


Score range 


written 


2 


2 


0.9S 


0.94 ■ 0.9S 


written 


2 


' 


0.87 


0.79 ■ 0.9S 


written 


1 


0 


0.44 




practical 


2 


0 


0.70 


0.64 -0.7S 



Primary 7 Recording and 
Presenting 

A total of six tasks, three written and three practi- 
cal, were included. 

Table II : P7 summary performance 



Level B 
Level C 
Level D 





No. of tasks 


No. "secure" 


Mean score 


Score range 


practical 


1 


1 


0.99 




written 


2 


2 


0.97 


0.97 - 0.98 


written 


1 


0 


0.58 




practical 


2 


2 


0.87 


both 0.87 



To meet 5-14 expectations, P7 pupils should have 
reached at least 0.8 on tasks at and below Level D. 
This they did on five of the six tasks (one Level D 
task involving a line graph. Teacup, failed to reach 
0 . 8 ). 

Secondary 2 Recording and 

Presenting 

A total of ten tasks, eight written and two practical, 
were included at S2. 

Table 12: S2 summary performance 



Level C 
Level D 
Level E 





No. of tasks 


No. "secure" 


Mean score 


Score range 


written 


2 


2 


0.97 


0.96 - 0.98 


written 


5 


2 


0.79 


0.64 - 0.95 


written 


1 


0 


0.4S 




practical 


2 


0 


0.S8 
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To meet 5-14 expectations, S2 pupils should have 
attained > 0.8 on all tasks set. While they did rea- 
sonably well with handling information in tables 
and graphs, S2 did not perform well on tasks which 
drew on knowledge of the conventional symbols 
and diagrams used in science. Both P7 and S2 pu- 
pils showed uncertainty in tasks involving line 
graphs. 

Comment 

Tasks tended to focus on inserting given informa- 
tion into tables, graphs etc. and were successfully 
completed by most pupils. In most respects per- 
formance at P4 and P7 was close to expected 5-14 
Levels. S2 pupils did not reach the expected Level. 
Where the task contained a line graph, perform- 
ance levels did not meet expectations (P7 and S2 
pupils only). S2 pupils also performed poorly on 
tasks which might be expected in the production of 
an investigation report: drawing the apparatus and 
instruments they used in practical contexts. 



S Interpreting 
and Evaluating 



The 5-14 Strand Interpreting and Evaluating had 
two sub-categories. Tasks from the first, Assign- 
ing Meaning, assessed targets such as identifying 
relationships, explaining, forming generalisations, 
drawing conclusions and predicting. Tasks from 
the second, Evaluating Evidence, focused on dis- 
tinguishing between inferences and observations. 

The ttaslkss wdiittcsini 

In the four tasks from the first sub-category, pupils 
were required to identify a relationship, pattern or 
trend from information presented in a graphical 
form. 

Example 9 : Bridgespan 



Lucy made a model bridge out of two blocks of wood and a 
piece of card. 




She measured the length and counted the number of 2p coins 
which the bridge could support. She did this several times. 

Here are her results: 



length (cm) 


number of 
2p coins 


10 


5 


12 


4 


14 


3 


16 


2 


18 


1 



What do you notice about the length of the bridge and the 
number of 2p coins which it could support? 



This Level B task asked pupils to identify an in- 
verse relationship — ‘the longer the bridge, the 
fewer coins it can support’. 

Most P4 pupils scored 0.38 and most P7 pupils 
scored 0.58. 

Most S2 pupils did significantly better with a score 
of 0.84 

TIhie ttffiskss piracttpcai 

The task Poles was included in the practical cir- 
cuits at Level D for P7 and S2 pupils. 

Example 10 : Poles 



On the table are 3 bar magnets. One of the bar magnets is 
wrapped in coloured paper. 

Hold the two unwrapped magnets together like this. 



H 


S 


S 


H 



What happens? 

Hold the two bar magnets together like this. 



s 


H 


s 


K 



What happens? 

Use one of the bar magnets to find out which end of the wrapped 
magnet is the "N" pole. 

I think the "N" end of the wrapped magnet is the end labelled 
Why do you think so? 



In the first part of this task, pupils had to note what 
happened as two magnets were placed as specified. 
In the second part they had to use the evidence to 
determine the correct pole of a third, covered, mag- 
net. 

Most P7 pupils scored 0.68, most S2 pupils 0.76. 
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Level B 
Level C 



Level B 
Level C 

Level D 



o 




Primary 4 Interpreting and 
Evaluating 



Secondary 2 Interpreting and 
Evaluating 



Twelve tasks, nine written and three practical were 
included in the P4 assessment materials. Table sum- 
marises performance levels. 

Table 14 : P4 performance summary 



Twenty-four tasks were included at S2; 20 written 
and four practical. Table 16 summarises the find- 
ings. 

Table 16: S2 summary performance 





No. of tasks 


No. "secure" 


Mean score 


Score range 


written 


4 


0 


0.52 


0.38 - 0.64 


written 


5 


0 


0.53 


0.47 - 0.61 


practical 


3 


0 


0.56 


0.50 - 0.68 



To meet 5-14 expectations for P4, the index scores 
for all Level B tasks should reach 0.8 or above and 
show some grasp at higher levels. P4 pupils did 
not quite attain scores consistent with Level B al- 
though they did show some understanding and skill 
in a majority of the tasks tackled. As in previous 
surveys, a significant number of P4 pupils made 
no attempt at questions requiring extended re- 
sponses. 

Primary 7 Interpreting and 
Evaluating 

Twenty tasks were included, 16 written and four 
practical. 



Table 15 : P7 summary performance 





No. of tasks 


No. "secure" 


Mean score 


Score range 


written 


3 


1 


0.70 


0.58 - 0.84 


written 


10 


1 


0.66 


0.48 - 0.80 


practical 


3 


1 


0.71 


0.65 ■ 0.80 


written 


3 


0 


0.42 


0.29 ■ 0.45 


practical 


1 


0 


0.68 





To meet 5- 14 expectations, most pupils should have 
reached 0.8 for all tasks at the Levels to D and show 
a degree of competence on those at Level E. Over- 
all, P7 pupils did not achieve the Level D expecta- 
tions. 




In order to meet ES 5-14 expectations, S2 pupils 
should have attained 0.8 on all tasks in the survey. 
Most pupils achieved this on Level B tasks, and all 
but one of the Level C scores were in the 0.6-0. 7 
range. Performance on Level D tasks was very 
varied but never reached 0.8. 

Comment 

Tasks in this Strand required pupils to identify re- 
lationships in data and to evaluate evidence from a 
variety of sources. At no stage did pupils produce 
scores which could be said to have met the appro- 
priate 5-14 Levels. As in previous surveys, there 
was clear evidence that pupils could extract spe- 
cific pieces of information from a range of formats 
and add information to tables, graphs etc. but could 
not interpret the messages conveyed by data pre- 
sented in these formats. 

In evaluating evidence and sorting observations 
from inferences, pupils again were operating be- 
low the Levels which were expected of them. As 
in all three previous AAP Science surveys, they 
‘jumped to conclusions’ by selecting likely expla- 
nations without considering whether the evidence 
justified the inference(s). 
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6 Practical 
Investigations 



While the assessment of individual Strands and 
Targets provides useful information on what chil- 
dren know and can do in aspects of science, it is 
when they try to integrate the various knowledge 
and skill bases in the context of extended investi- 
gations that they reflect the demands of ‘doing sci- 
ence’. 

The tasks 

There were two investigations, Race Track and Ris- 
ing Damp. The first was based on motion down an 
inclined plane, the second was concerned with the 
rate at which liquids rise through different grades 
of paper. These were relatively closed investiga- 
tions where pupils were presented with a specific 
question to answer which required them to work 
with practical apparatus, to gather evidence and, 
on the basis of that evidence, reach a conclusion 

Primary 4 

P4 pupils performed better on Race Track than Ris- 
ing Damp. The purpose of Rising Damp and the 
specific content appears to have been less accessi- 
ble and relevant to them. Approximately 66 per 
cent of pupils in the sample displayed a satisfac- 
tory level of manipulative skills. While many pu- 
pils did undertake fair tests on Race Track, this 
appeared to be by good fortune rather than design. 
The variables to be controlled in Rising Damp were 
less obvious and only around 25 per cent of P4 
pupils made ‘fair tests’ of the conditions in this in- 
vestigation. P4 pupils appeared, both in tackling 
the investigation and during the probing at the end, 
unaware of the need to take deliberate steps to en- 
sure fair tests. 

As in previous years, very few pupils recorded re- 
sults. Virtually all came to a conclusion at the end 
of the investigation, although some had been car- 
ried out so haphazardly that no conclusion was jus- 
O ified. Only a small proportion acknowledged this 



when given the opportunity to reflect critically upon 
the activity and suggest changes. 

Primary 7 

Virtually all pupils appeared to understand the task 
initially and the majority proceeded in a system- 
atic manner to test the cars in Race Track and grades 
of paper in Rising Damp. 

It is difficult to interpret the low percentage of pu- 
pils judged to be satisfactory in handling the appa- 
ratus/instruments on Race Track although it may 
be due to the Field Officers holding higher expec- 
tations of the P7 pupils on the same task and find- 
ing them wanting. (The same field officers assessed 
P4 and P7 but not S2.) 

As at P4, those tackling Race Track were much 
more successful in controlling for fair tests than 
those on Rising Damp. Similarly, the majority re- 
corded results in Race Track, and far fewer did in 
Rising Damp. Half of the pupils acknowledged that 
there were ways in which the method used could 
be improved. 

Secondary 2 

Almost all S2 pupils understood Race Track and 
proceeded in a systematic way to test the condi- 
tions. While most pupils understood the purpose 
of Rising Damp, a substantially smaller percent- 
age undertook a systematic investigation. 

On Race Track, the majority of pupils controlled 
the relevant variables in a deliberate, systematic 
way, measured accurately the distance the block 
travelled and recorded the results on the pupil book- 
let. They drew valid conclusions and appeared 
aware of how such investigations should be con- 
ducted. 

In contrast, while almost all appeared to understand 



what was required in Rising Damp, around 61 per 
cent controlled potentially confounding variables 
and just under 50 percent made accurate measure- 
ments of the dependent variable. 

Approximately 75 per cent recorded the results of 
their tests and most reached a conclusion on the 
grade of paper through which the water rose more 
quickly. 

Given the opportunity to reflect, between 40 per 
cent and 50 per cent of those working on Rising 
Damp made relevant suggestions for improvements, 
suggesting that they were aware of deficiencies in 
the approach taken. 

Comment 

It should be noted that the experiences of the team 
and feedback from Field Officers indicated that a 
significant proportion of pupils at all three stages, 
but more in particularly the primary sector, were 
unfamiliar with practical activities in science. How- 
ever, the pupils set to the tasks readily and demon- 
strated considerable learning in the course of the 
investigation. This is reflected in the nature of the 
many suggestions which they made for improve- 
ment, indicating that their participation had been 
as much a learning experience as an opportunity to 
demonstrate acquired knowledge and skills. 

Until such extended practical activities, in a range 
of formats, become a commonplace part of science 
learning and teaching in schools, assessment of in- 
tegrated tasks such as these can only indicate gen- 
eral patterns of performance in broad categories of 
knowledge and skill. 



7 Pupils' Views 
on Learning Science 



The questionnaire to pupils had three sections. The 
first asked P4, P7 and S2 pupils how confident they 
felt about their own knowledge in a number of topic 
areas and to identify the main source of that learn- 
ing. The second asked them to show their prefer- 
ence for studying different topics in science and 
the third sought information on the frequency with 
which they experienced a range of activities in the 
science classroom and how much they enjoyed 
these. The last section was only set at P7 and S2. 

Scientific knowledge and sources 
of learning 

At all three stages, pupils were most confident in 
their understanding of topics which drew on the 
key features of AOl : Living Things and the Proc- 
esses of Life. Generally speaking, P4 pupils were 
more confident in their own scientific knowledge 
than those in the older groups. The figures may be 
a reflection of the content provided at different 
stages of primary and early secondary school and, 
if so, this would indicate that topics from A02: 
Energy and Forces and AOS: Earth and Space are 
only really contributing to pupils’ understanding in 
the upper primary and early secondary years. 

A worrying feature is the growing emphasis on the 
school as a key source of scientific knowledge as 
pupils progress from P4 to S2, at the expense of 
other sources, in the views of pupils. Other key 
sources such as parents, television, museums and 
books diminish as children grow older, according 
to their stated judgements. When pupils leave 
school their main source of information is closed 
and other sources have gone into disuse. This does 
not bode well for the notion of a scientifically liter- 
ate population. 

Favourite topics in science 

When pupils were asked to rank their preference 
Q For a number of topics in science, boys and girls 
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demonstrated different patterns of response. Firstly, 
whatever the age group, the girls were much more 
in agreement in their choices than were the boys. 
Secondly, the pattern of preferences amongst P4 
girls was not greatly different from that given by 
the P7 and the S2 girls; the boys’ preferences were 
more varied and showed a clear shift in their pref- 
erences from the biological sciences at P4 to the 
physical sciences at S2. 

Learning activities in science 

Only P7 and S2 pupils were involved in this sec- 
tion of the questionnaire and their responses corre- 
sponded well with the (less detailed) reports from 
schools. In particular, there was evidence of a 
greater emphasis on whole class teaching in sec- 
ondary schools than in the primary sector. 

Both samples of pupils reported that they rarely had 
the opportunity to follow up their own questions in 
school and that teacher explanations/demonstra- 
tions were amongst the more frequent forms of ac- 
tivity. Patterns of preference were similar in P7 
and S2, with problem solving, the use of apparatus 
and watching television/video programmes 
amongst the more popular activities. 

In general, both boys and girls were more positive 
than negative in their responses although at P7 there 
was some evidence that boys were more polarised 
in their views of science activities, i.e. either posi- 
tive or negative, whereas girls were more likely to 
remain neutral. 

Commemt 

At all three stages, pupils expressed the greatest 
confidence in AOl: Living Things and the Proc- 
esses of Life although the gap between the Attain- 
ment Outcomes narrowed in P7 and again in S2, 
possibly a result of the increased teaching focus on 
physics and chemistry topics as pupils get older. 
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S2 pupils are heavily dependent on the school as 
their main source of information about science top- 
ics and concepts. Alternative information routes 
do not seem to be well established to sustain inter- 
est and knowledge when pupils leave school, or 
give up science. 

The opportunity to follow up their own investiga- 
tions in class appears more likely to be afforded to 
S2 pupils than P7 pupils. 
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8 Information 
from Schools 



Questionnaires were sent to every school in the 
sample (96 secondary and 215 primary) requesting 
general background information on each school’s 
policy for the learning and teaching of science at 
the stages concerned. In addition, information was 
sought on the materials and approaches used and 
teachers’ views on the important aspects of science 
teaching. 

A policy for science 

Just over half of the primary schools reported that 
they did have a policy for science while approxi- 
mately one third said that they did not. (The 1993 
figures were very similar.) In primary schools, 
where there was a policy it tended to be one where 
science was an integral part of environmental stud- 
ies. A small, but growing, percentage of schools 
regarded science as the main focus for topic study 
or as a subject in its own right. Most secondary 
schools reported that science was taught as ‘inte- 
grated science’ in S2 while around one fifth said 
that it was taught as ‘separate sciences’. 

Only half of the primary schools and an even 
smaller proportion of secondary schools stated that 
the policy reflected the 5-14 Environmental Stud- 
ies guidelines. 
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9 Learning and 
Teaching Science 



Primary Schools 

The 5“ 14 guidelines imply that around 7.5 per cent 
of class time should be devoted to science. 

In approximately 30 per cent of primary schools 
surveyed, 5 per cent to 10 per cent of the timetable 
was reported to be allocated to science. A very small 
number allocated more than this, with the remain- 
der giving < 5 per cent. This represents a notice- 
able increase in the proportion of primary schools 
allocating between 5 per cent and 10 per cent of the 
time to science since 1993. This can probably be 
attributed, at least in part, to the influence of the ES 
5-14 guidelines. However, approximately 40 per 
cent of primary schools are allocating less than the 
recommended time. 

Primary teachers identified pressure of time as the 
biggest obstacle to increasing the time allocated to 
science. Lack of resources and lack of accommo- 
dation were also significant problems; only 26 per 
cent of schools reported being ‘well equipped’ with 
practical or commercial materials. 

Compared with 1993, fewer regarded teachers’ at- 
titudes/qualifications etc. as a barrier to increased 
provision (36 per cent in 1996; 70 per cent in 1993). 
However, the evidence suggests that there is still 
some way to go in providing adequate support for 
teachers in implementing science in line with ES 
5-14. A substantial number of teachers reported no 
in-service or P.A.T. provision for science and only 
21 per cent indicated that support was provided by 
other means. 



Secondary Schools 

The 5-14 guidelines recommend a minimum allo- 
cation of 10 per cent (approximately 2 hours 40 
minutes) to science in the secondary school; 99 per 
cent of schools are providing a time close to or in 
excess of this recommendation. 

There were clear differences in the way teaching 
and learning was organised in primary and 
secondary classes. In primary schools there was 
more emphasis on group-based working than 
whole- class teaching. In secondaries, although 
many teachers reported using group work and/or 
individualised learning, there was more emphasis 
on whole-class teaching than in primary schools. 

Assessment was carried out frequently in almost 
all schools and most often by the ‘end of topic 
test’. Assessment appears to be used primarily 
for summative purposes; no school reported the 
use of assessment for specifically diagnostic 
purposes. 

In both primary and secondary sectors the em- 
phasis on understanding of basic science con- 
cepts has increased since the previous survey. In 
the 1993 primary sample, just under half opted 
for this goal whereas almost three quarters did so 
in 1996. In secondaries the corresponding 
figures were significantly higher in both surveys. 
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The 5-14 Environmental Studies guidelines do 
appear to have had an influence on practice, but 
there is still some way to go before they are fully 
operational in all schools, particularly in the 
primary sector. This may be a reflection of 
structural differences in primary and secondary, 
with primaries having to prioritise developments 
and phase in changes over a longer period. 

While the specific learning goals selected were 
similar to previous surveys in both primary and 
secondary sectors, there was an increased empha- 
sis on ‘understanding of basic science concepts’ 
in both. This may well be, at least in part, due to 
the influence of the ES 5 - 14 document. 

The feedback from schools indicates that, overall, 
there has been some change in policy and prac- 
tice ot be said to reflect the full implementation 
of the guidelines on science and rather should be 
considered as baseline measures of attainment 
against which performance in future surveys can 
be compared. 



O 

ERIC 



22 



27 



10 Performance 
Comparisons 



Performance over time 

Across the three surveys since 1990: 

• P4 performance has risen on written but not 
on practical tasks 

• P7 performance has remained fairly steady 
on both components 

• S2 performance has fallen somewhat on writ- 
ten tasks although the position is less clear 
with regard to practical tasks. 

The 1993-96 data was analysed to determine 
whether these changes were related to gender. At 
P4, all significant differences between 1993 and 
1996 were in a positive direction, with marginally 
greater improvement shown by girls, across the 
common tasks. At P7, positive changes consider- 
ably outweighed negative ones, with little differ- 
ence between the performance levels of boys and 
girls. The pattern was reversed however at S2 
where the majority of significant differences point 
to a decline in performance marginally greater for 
boys than girls. 

Performance between stages 

Pupils at P7 and S2 stages produced performances 
significantly superior to those at P4 on all of the 
common tasks. Comparisons of the P7 and S2 per- 
formance levels are less clear cut, with the older 
group producing significantly better success rates 
than P7 on 67 percent of the common written tasks 
and 56 per cent of the practical ones. 

These patterns are in line with those of earlier sur- 
veys, indicating that while pupils appear to make 
significant developments in the knowledge, skills 
and processes assessed across the P4 to P7 stages, 
the difference in performance from P7 to S2 does 
not appear to reflect the two years of specialist sci- 
ence teaching which they have experienced. 



In the longitudinal study which followed a sub-sam- 
ple of P4 pupils from 1993 through to 1996, the 
main findings indicate that, generally, those pupils 
who performed above average in 1993 continued 
to do so in 1996, with a slight indication of an in- 
creasing gap between the high and low achievers 
from P4 to P7. 



1 1 Issues 
and Implications 



The main findings and the implications for the is a particularly enjoyable aspect of school 

learning, teaching and assessment of science in the science. 

primary and early secondary stages are as follows. ® The poor performance of S2 pupils in 

1996, and its decline over time, indicates 

i. Performance in 1996 the need for a major review of the teaching 

The main conclusions are that: and learning of science in the first two 

® performance levels were good at P4 and years of secondary school, 

fair at P7, although specific Key Features 
and Strands showed signs of weakness 
® performance levels at S2 gave cause for 
concern across all aspects tested, except 
perhaps handling information 
® the two younger groups had improved or 
maintained performance levels while S2 
pupils showed a slight decline 
® differences in performance levels between 
pupils P7 and S2 gave concern over 
learning and teaching in the first two years 
of secondary school 

• gender differences in performance levels 
along traditional lines were evident, with 
implications for teaching, learning and 
assessment 

• a generally positive attitude to science was 
indicated at all stages, though this tended 
to wane as pupils progressed through 
school. 

ii. Policy and Provision 

The main messages are that: 

ES 5- 14 guidelines on science are beginning 
to make their presence felt, at least in pri- 
mary schools. Progress is slow however. 

Obstacles to effective implementation — 
not least time and resources — have been 
identified by teachers. 

Staff development opportunities are 
required both locally and nationally. 

Teachers place increasing emphasis on 
knowledge and understanding in science 
teaching and learning. 

Opportunities for ‘hands on’ science are 
relatively rare although pupils indicate this 
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